05 - Automation-System | DeepWiki

Relevant source files

CLAUDE.md

scripts/README.md

scripts/ntfy-godeep-automation.sh

Purpose and ScopeLink copied!

The Automation System is the event-driven pipeline that listens for GitHub installation notifications and automatically provisions private repository mirrors in the owner's GitHub account. This system bridges the gap between customer repository selection and owner repository access by cloning customer repositories and creating private copies under the klaudioz GitHub organization.

Important: This system automates repository cloning and access provisioning only. It does NOT automate documentation generation. For information about manual documentation generation workflow, see Manual Repository Access Workflow. For information about the ntfy notification sources, see ntfy Integration & Message Broker.

Sources: scripts/ntfy-godeep-automation.sh L1-L152

scripts/README.md L1-L81

CLAUDE.md L7-L9

System ArchitectureLink copied!

The automation system implements an event-driven architecture where external events (GitHub installations) trigger automated repository provisioning workflows. The system uses ntfy.sh as a message broker to decouple event sources from event consumers, enabling asynchronous processing without requiring a persistent application server or database.

Automation Pipeline OverviewLink copied!

Key Components:

Component	Type	Location	Purpose
`ntfy.sh`	External Service	Topic: `topic-XXXXXXXX`	Message broker for asynchronous event delivery
`ntfy-godeep-automation.sh`	Bash Script	scripts/ntfy-godeep-automation.sh	Local automation script that subscribes to ntfy
`ntfy-godeep-automation-remote.sh`	Bash Script	(mentioned but not in files)	Remote variant for server deployment
`/api/admin/generate-token`	API Endpoint	Admin API	Generates installation access tokens
`~/godeep-clones/`	Directory	Local filesystem	Temporary storage for cloned repositories
`osascript`	System Command	macOS	Displays native notifications

Sources: scripts/ntfy-godeep-automation.sh L1-L152

scripts/README.md L24-L49

Event Processing PipelineLink copied!

The automation pipeline processes GitHub installation events through a series of discrete stages, transforming a notification payload into a provisioned private repository.

Pipeline Flow DiagramLink copied!

Pipeline Stages:

Event Reception: ntfy subscribe blocks and receives JSON events from ntfy.sh
Message Parsing: jq extracts message and title fields
Event Filtering: grep checks if title matches GitHub installation pattern
ID Extraction: Regex 'Installation ID: [0-9]*' extracts numeric installation ID
Token Generation: curl POST to admin API with installation ID and password
Response Parsing: jq extracts token, repository details, and customer email
Name Formatting: Email transformation (@ → _, . → _, lowercase)
Repository Cloning: git clone with token-based authentication
Git Reinitialization: Remove original .git, create new commit with customer attribution
Private Repo Creation: gh repo create with --private flag under klaudioz org
Cleanup: rm -rf removes local clone to conserve disk space
Notification: osascript displays macOS notification with status

Sources: scripts/ntfy-godeep-automation.sh L39-L151

scripts/README.md L38-L51

Script Implementation DetailsLink copied!

Environment Variables and ConfigurationLink copied!

The automation script reads configuration from multiple sources:

Variable	Source	Line Numbers	Purpose
`TOPIC`	Hardcoded	scripts/ntfy-godeep-automation.sh L6	ntfy.sh topic name
`ADMIN_URL`	Hardcoded	scripts/ntfy-godeep-automation.sh L7	Admin API endpoint URL
`CLONE_DIR`	Hardcoded	scripts/ntfy-godeep-automation.sh L8	Base directory for clones
`SCRIPT_DIR`	Computed	scripts/ntfy-godeep-automation.sh L11	Directory containing script
`PROJECT_ROOT`	Computed	scripts/ntfy-godeep-automation.sh L13	Parent directory of scripts/
`ENV_FILE`	Computed	scripts/ntfy-godeep-automation.sh L16	Path to `.env` file
`ADMIN_PASSWORD`	`.env` file	scripts/ntfy-godeep-automation.sh L23	Admin authentication password

Environment File Parsing:

# Extract ADMIN_PASSWORD from .envADMIN_PASSWORD=$(grep "^ADMIN_PASSWORD=" "$ENV_FILE" | cut -d '=' -f2- | tr -d '"' | tr -d "'")

The script uses grep, cut, and tr to extract the password value, removing quotes and single quotes. This parsing is brittle and assumes the password does not contain = characters.

Sources: scripts/ntfy-godeep-automation.sh L6-L28

Message Format and ParsingLink copied!

ntfy JSON Message Structure

{  "message": "Installation ID: 12345678\nUsername: customer-username\nEmail: customer@example.com\nMatch ID: abc123def456",  "title": "GitHub Connected"}

Parsing Implementation

The parsing uses grep -o with regex patterns to extract numeric IDs from unstructured text. This approach is fragile—if the message format changes, parsing breaks.

Sources: scripts/ntfy-godeep-automation.sh L39-L54

API IntegrationLink copied!

Token Generation Request

The script calls the admin API to generate an installation access token:

response=$(curl -s -X POST "$ADMIN_URL" \    -H "Content-Type: application/json" \    -d "{\"installationId\":\"$installation_id\",\"password\":\"$ADMIN_PASSWORD\"}")

Request Structure:

Method: POST
Endpoint: https://www.godeep.wiki/api/admin/generate-token
Headers: Content-Type: application/json
Body: {"installationId":"12345678","password":"secret"}

Response Structure:

{  "token": "ghs_...",  "expiresAt": "2024-01-01T12:00:00Z",  "repositories": [    {      "name": "my-repo",      "full_name": "customer/my-repo",      "private": true    }  ],  "user": {    "email": "customer@example.com",    "username": "customer-username"  }}

Response Parsing

token=$(echo "$response" | jq -r '.token // empty')repo_full_name=$(echo "$response" | jq -r '.repositories[0].full_name // empty')repo_name=$(echo "$response" | jq -r '.repositories[0].name // empty')customer_email=$(echo "$response" | jq -r '.user.email // .user.username // empty')

The script uses jq with the // empty operator to provide fallback values, and chains .user.email // .user.username to use username if email is unavailable.

Sources: scripts/ntfy-godeep-automation.sh L62-L71

Repository Naming and FormattingLink copied!

Email-to-Filename TransformationLink copied!

The script transforms customer email addresses into filesystem-safe repository names:

if [ -n "$customer_email" ]; then    email_prefix=$(echo "$customer_email" | sed 's/@/_/g; s/\./_/g' | tr '[:upper:]' '[:lower:]')    formatted_repo_name="${email_prefix}-${repo_name}"else    formatted_repo_name="$repo_name"fi

Transformation Rules:

Input Character	Output Character	Command
`@`	`_`	`sed 's/@/_/g'`
`.`	`_`	`sed 's/\./_/g'`
Uppercase	Lowercase	`tr '[:upper:]' '[:lower:]'`

Examples:

Customer Email	Repository Name	Formatted Name
`john@example.com`	`my-app`	`john_example_com-my-app`
`Alice.Smith@Corp.IO`	`backend`	`alice_smith_corp_io-backend`
(empty)	`frontend`	`frontend`

This naming convention enables tracking and prevents collisions when multiple customers share the same repository name.

Sources: scripts/ntfy-godeep-automation.sh L78-L84

Git Operations WorkflowLink copied!

Clone, Reinitialize, and Push SequenceLink copied!

Critical Implementation Details:

Token-Based Clone: Uses https://x-access-token:TOKEN@github.com/ format for authentication
History Removal: rm -rf .git removes customer's commit history for privacy
Single Commit: Creates one commit with customer email in message for attribution
Private Repository: --private flag ensures repo is not publicly accessible
Atomic Push: --source=. --push creates repo and pushes in single operation
Immediate Cleanup: rm -rf runs regardless of push success to prevent disk space exhaustion

Sources: scripts/ntfy-godeep-automation.sh L86-L131

Clone Command DetailsLink copied!

git clone "https://x-access-token:${token}@github.com/${repo_full_name}.git" "$clone_path"

URL Structure:

Protocol: https://
Authentication: x-access-token:TOKEN@
Domain: github.com
Path: /customer-org/repo-name.git
Destination: ~/godeep-clones/formatted_repo_name

The x-access-token username is a GitHub convention for installation access tokens. Any username works, but x-access-token is the documented standard.

Sources: scripts/ntfy-godeep-automation.sh L90

Repository Creation CommandLink copied!

gh repo create "klaudioz/$formatted_repo_name" --private --source=. --push

GitHub CLI Flags:

Flag	Value	Purpose
`--private`	(boolean)	Create private repository (not public)
`--source=.`	Current directory	Initialize from local filesystem
`--push`	(boolean)	Push commits immediately after creation

This command requires the gh CLI to be authenticated. The script assumes gh auth login has been run previously.

Sources: scripts/ntfy-godeep-automation.sh L106

Notification SystemLink copied!

macOS Notification IntegrationLink copied!

The script uses osascript to display native macOS notifications at key pipeline stages:

Notification Triggers:

Stage	Notification Title	Notification Body	Line Numbers
Start	"GoDeep.wiki Automation"	"Processing Installation ID: 12345678"	scripts/ntfy-godeep-automation.sh L59
Clone Success	"✅ GoDeep.wiki - Clone Complete"	"Repository cloned to /path/to/clone"	scripts/ntfy-godeep-automation.sh L94
Create Success	"✅ GoDeep.wiki - Complete!"	"Repository created: klaudioz/name"	scripts/ntfy-godeep-automation.sh L117
Clone Failure	"❌ GoDeep.wiki Error"	"Clone failed for customer/repo"	scripts/ntfy-godeep-automation.sh L134
Create Failure	"❌ GoDeep.wiki Error"	"Failed to create repo: name"	scripts/ntfy-godeep-automation.sh L129
Token Failure	"❌ GoDeep.wiki Error"	"Failed to process Installation ID: 12345678"	scripts/ntfy-godeep-automation.sh L139
Other Events	(event title)	(event message)	scripts/ntfy-godeep-automation.sh L147

osascript Command Format:

osascript -e "display notification \"$body\" with title \"$title\""

This only works on macOS. The script will fail on Linux or Windows without modification.

Sources: scripts/ntfy-godeep-automation.sh L59-L147

Error Handling and CleanupLink copied!

Cleanup StrategyLink copied!

The script implements aggressive cleanup to prevent disk space exhaustion:

# Return to parent directory before deletioncd - > /dev/null# Delete local clone to save spaceecho "🗑️  Cleaning up local clone..."rm -rf "$clone_path"echo "✅ Local clone deleted: $clone_path"

Key Behaviors:

Always Cleanup: Runs rm -rf regardless of success or failure
Directory Exit: cd - before deletion to avoid deleting current working directory
Silent Navigation: > /dev/null suppresses directory change output
No Confirmation: rm -rf runs without prompt (dangerous if clone_path is miscalculated)

Potential Issues:

If clone_path variable is empty, rm -rf "" could delete unexpected files
If cd fails, script continues with cleanup from wrong directory
No disk space checks before cloning large repositories

Sources: scripts/ntfy-godeep-automation.sh L109-L127

Error Exit ConditionsLink copied!

Pre-Flight Checks:

Check	Failure Behavior	Exit Code	Line Numbers
`.env` file exists	Print error, exit	1	scripts/ntfy-godeep-automation.sh L17-L20
`ADMIN_PASSWORD` found	Print error, exit	1	scripts/ntfy-godeep-automation.sh L25-L28

Runtime Behavior:

No exit on API failures—logs error and waits for next notification
No exit on clone failures—cleans up and continues
Ctrl+C is the only way to stop script gracefully
Script blocks indefinitely on ntfy subscribe command

Sources: scripts/ntfy-godeep-automation.sh L17-L28

Deployment and OperationLink copied!

PrerequisitesLink copied!

The automation script requires the following tools to be installed and configured:

Tool	Purpose	Installation Check	Configuration
`ntfy`	Subscribe to ntfy.sh notifications	`which ntfy`	N/A (uses public service)
`jq`	Parse JSON responses	`which jq`	N/A
`git`	Clone repositories	`which git`	User must be authenticated
`gh`	GitHub CLI for repo creation	`which gh`	Must run `gh auth login`
`curl`	HTTP requests to admin API	`which curl`	N/A (built-in on macOS)
`osascript`	macOS notifications	`which osascript`	macOS only

Installation Commands (macOS with Homebrew):

brew install ntfybrew install jqbrew install ghgh auth login  # Required for gh repo create

Sources: scripts/README.md L66-L80

Running the ScriptLink copied!

Foreground Execution

# From project root./scripts/ntfy-godeep-automation.sh

Output Example:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🚀 GoDeep.wiki Automation Started
Listening for new GitHub installations...
Clones will be saved to: /Users/username/godeep-clones
Press Ctrl+C to stop

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🔔 Received: GitHub Connected
✓ GitHub Installation detected!
🔑 Installation ID: 12345678
🔐 Generating access token...
✓ Token generated successfully
📦 Repository: customer/my-app
👤 Customer: customer@example.com
📂 Cloning to: /Users/username/godeep-clones/customer_example_com-my-app
✓ Clone successful!

📤 Pushing to your personal GitHub account...
✅ Repository created: klaudioz/customer_example_com-my-app
🗑️  Cleaning up local clone...
✅ Local clone deleted: /Users/username/godeep-clones/customer_example_com-my-app

Sources: scripts/ntfy-godeep-automation.sh L33-L151

Background Execution

# Run in background with loggingnohup ./scripts/ntfy-godeep-automation.sh > ~/ntfy-godeep.log 2>&1 &# View logs in real-timetail -f ~/ntfy-godeep.log# Stop background processpkill -f ntfy-godeep-automation# Find process IDps aux | grep ntfy-godeep-automation

The nohup command prevents the script from terminating when the terminal session ends. Output and errors are redirected to ~/ntfy-godeep.log.

Sources: scripts/README.md L19-L21

scripts/README.md L55-L63

Remote Automation VariantLink copied!

The codebase mentions a remote automation variant (ntfy-godeep-automation-remote.sh) that is designed for server deployment, but this file is not included in the provided sources. Based on the README documentation, the remote variant likely differs in:

Notification System: Removes osascript calls (not available on Linux servers)
Clone Directory: Uses different base path for server filesystem
Logging: May use syslog or file-based logging instead of stdout
Process Management: May include systemd service integration

Expected Differences:

Feature	Local Script	Remote Script (assumed)
Notifications	`osascript`	None or webhook-based
Clone Path	`~/godeep-clones`	`/var/godeep-clones` or similar
Logging	stdout	File or syslog
Daemon	Manual `nohup`	systemd service
OS	macOS only	Linux (Ubuntu/Debian)

To use the remote variant, operators would need to adapt the script for their server environment by removing macOS-specific commands and adjusting paths.

Sources: scripts/README.md L1-L81

Security ConsiderationsLink copied!

Token Exposure RisksLink copied!

The automation script handles sensitive credentials:

Credential	Storage	Exposure Risk	Mitigation
`ADMIN_PASSWORD`	`.env` file	File system access	File permissions (600)
Installation access token	Memory only	Process inspection	Short lifespan (1 hour)
Customer repository URLs	Memory only	Process inspection, logs	Cleared after use
`gh` authentication	`~/.config/gh/`	File system access	GitHub CLI manages securely

Logged Sensitive Data:

The script logs sensitive information to stdout:

echo "🔑 Installation ID: $installation_id"echo "👤 Customer: $customer_email"echo "📂 Cloning to: $clone_path"

If running with > ~/ntfy-godeep.log, these values are written to disk. The log file should have restricted permissions:

chmod 600 ~/ntfy-godeep.log

Sources: scripts/ntfy-godeep-automation.sh L56-L88

Git Credential CachingLink copied!

The git clone command uses tokens in URLs:

git clone "https://x-access-token:${token}@github.com/${repo_full_name}.git"

Risks:

Tokens appear in shell history (if commands are typed manually)
Tokens appear in process list (visible via ps aux)
Tokens may be cached by git credential helpers

Mitigation:

Script uses variables, not literal tokens in commands
Tokens expire after 1 hour
Clone directories are deleted immediately after use

Sources: scripts/ntfy-godeep-automation.sh L90

Monitoring and TroubleshootingLink copied!

Common Failure ScenariosLink copied!

Authentication Failures

Symptoms:

{"error": "Authentication failed"}

Causes:

ADMIN_PASSWORD in .env doesn't match Vercel environment variable
API endpoint not deployed with latest changes
Vercel function cold start timeout

Resolution:

Verify password in .env matches Vercel: grep ADMIN_PASSWORD .env
Redeploy API: vercel --prod
Check Vercel logs for API errors

Sources: scripts/README.md L73-L75

Clone Failures

Symptoms:

❌ Clone failed

Causes:

Token expired (1-hour limit exceeded)
Network connectivity issues
Repository no longer accessible (permissions revoked)
Repository is empty (no commits)

Resolution:

Check token timestamp—regenerate if >1 hour old
Verify network: curl -I https://github.com
Manually test token: curl -H "Authorization: token $TOKEN" https://api.github.com/installation/repositories

Sources: scripts/README.md L77-L80

Repository Creation Failures

Symptoms:

❌ Failed to create GitHub repository

Causes:

gh CLI not authenticated
Repository name already exists in klaudioz org
Network timeout during push
GitHub rate limit exceeded

Resolution:

Re-authenticate: gh auth login
Check existing repos: gh repo list klaudioz
Delete conflicting repo: gh repo delete klaudioz/repo-name
Wait if rate limited (resets hourly)

Sources: scripts/ntfy-godeep-automation.sh L106-L130

Diagnostic CommandsLink copied!

# Check script is runningps aux | grep ntfy-godeep-automation# View recent logs (if background)tail -n 100 ~/ntfy-godeep.log# Test ntfy connectivityntfy subscribe topic-XXXXXXXX --poll# Test admin API manuallycurl -X POST https://www.godeep.wiki/api/admin/generate-token \  -H "Content-Type: application/json" \  -d '{"installationId":"12345678","password":"YOUR_PASSWORD"}'# Check clone directoryls -lah ~/godeep-clones/# Verify gh authenticationgh auth status# Check disk spacedf -h ~/godeep-clones/

Sources: scripts/README.md L55-L80

Performance CharacteristicsLink copied!

Resource ConsumptionLink copied!

Resource	Usage Pattern	Typical Values
CPU	Spikes during clone/push	5-30% per operation
Memory	Stable, increases during clone	50-200MB baseline, +repo size
Disk	Temporary, cleared after each repo	Peak = largest repo size, ~0 steady state
Network	Bursty during clone/push	Download + upload = 2× repo size
Process Count	1 main process + child processes	1-5 concurrent

Timing Estimates:

ntfy event reception: <1 second (real-time)
API token generation: 1-3 seconds
Git clone: 5-60 seconds (depends on repo size)
Git operations (init/add/commit): 1-5 seconds
GitHub repo creation + push: 5-30 seconds
Total per installation: 15-100 seconds

Sources: scripts/ntfy-godeep-automation.sh L1-L152

Concurrency LimitationsLink copied!

The script processes notifications sequentially:

ntfy subscribe "$TOPIC" | while read -r line; do    # Blocking operations...done

Implications:

Only one repository processed at a time
New notifications queued if processing takes >notification interval
Long-running operations block subsequent events
No parallel cloning of multiple repositories

Bottlenecks:

Git clone speed (network-bound)
GitHub API rate limits (5000 requests/hour)
Disk I/O during large repository operations
Single-threaded bash execution

To process multiple installations concurrently, the script would need to be rewritten to spawn background processes or use a job queue system.

Sources: scripts/ntfy-godeep-automation.sh L39-L151

Refresh this wiki

Last indexed: 23 November 2025 (922b35)

Automation System
Purpose and Scope
System Architecture
Automation Pipeline Overview
Event Processing Pipeline
Pipeline Flow Diagram
Script Implementation Details
Environment Variables and Configuration
Message Format and Parsing
API Integration
Repository Naming and Formatting
Email-to-Filename Transformation
Git Operations Workflow
Clone, Reinitialize, and Push Sequence
Clone Command Details
Repository Creation Command
Notification System
macOS Notification Integration
Error Handling and Cleanup
Cleanup Strategy
Error Exit Conditions
Deployment and Operation
Prerequisites
Running the Script
Remote Automation Variant
Security Considerations
Token Exposure Risks
Git Credential Caching
Monitoring and Troubleshooting
Common Failure Scenarios
Diagnostic Commands
Performance Characteristics
Resource Consumption
Concurrency Limitations

Ask Devin about godeep.wiki-jb

Syntax error in text

mermaid version 11.4.1

05 - Automation-System

Purpose and ScopeLink copied!

System ArchitectureLink copied!

Automation Pipeline OverviewLink copied!

Event Processing PipelineLink copied!

Pipeline Flow DiagramLink copied!

Script Implementation DetailsLink copied!

Environment Variables and ConfigurationLink copied!

Message Format and ParsingLink copied!

ntfy JSON Message Structure

Parsing Implementation

API IntegrationLink copied!

Token Generation Request

Response Parsing

Repository Naming and FormattingLink copied!

Email-to-Filename TransformationLink copied!

Git Operations WorkflowLink copied!

Clone, Reinitialize, and Push SequenceLink copied!

Clone Command DetailsLink copied!

Repository Creation CommandLink copied!

Notification SystemLink copied!

macOS Notification IntegrationLink copied!

Error Handling and CleanupLink copied!

Cleanup StrategyLink copied!

Error Exit ConditionsLink copied!

Deployment and OperationLink copied!

PrerequisitesLink copied!

Running the ScriptLink copied!

Foreground Execution

Background Execution

Remote Automation VariantLink copied!

Security ConsiderationsLink copied!

Token Exposure RisksLink copied!

Git Credential CachingLink copied!

Monitoring and TroubleshootingLink copied!

Common Failure ScenariosLink copied!

Authentication Failures

Clone Failures

Repository Creation Failures

Diagnostic CommandsLink copied!

Performance CharacteristicsLink copied!

Resource ConsumptionLink copied!

Concurrency LimitationsLink copied!

On this page